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INTRODUCTION 

The  major  goals  of  this  continuing  project  are  the  establishment  of  a  telemedicine  system  for  comprehensive 
diabetes  management  and  the  assessment  of  diabetic  retinopathy  that  provides  increased  access  for  diabetic 
patients  to  appropriate  care,  that  centralizes  the  patients  in  the  care  process,  that  empowers  the  patient  to  better 
manage  their  disease,  that  can  be  performed  in  a  cost  effective  manner,  and  that  maintains  the  high  standard  of 
care  required  for  the  appropriate  management  of  diabetic  patients. 

The  aim  of  this  continuation  will  be  to  perform  the  appropriate  clinical  validation,  cost  efficiency,  and  risk 
benefit  studies  associated  with  the  use  of  the  recently  developed  Comprehensive  Diabetes  Management 
Program  (CDMP)  and  the  Joslin  Vision  Network  (JVN)  Eye  Health  Care  Program  that  is  now  a  module  of  the 
CDMP.  This  proposal  will  also  have  a  secondary  focus  on  continuing  technology  centric  research  associated 
with  the  development  of  new  modules  for  the  CDMP,  and  development  and  validation  of  computer  algorithms 
designed  to  automate  detection  of  retinal  lesions  developed  during  the  course  of  retinopathy  development. 

The  need  for  diabetes  disease  management  is  driven  by  the  knowledge  that  diabetes  is  not  currently  curable,  but 
it  is  treatable,  and  its  complications  are  preventable.  The  primary  goal  of  treatment  is  to  manage  diabetes  to  live 
a  healthy  life.  In  general,  the  traditional  physician-centered,  episodic,  acute-care  model  is  not  designed  to  care 
for  large  numbers  of  diabetic  patients  (2,200  new  cases  diagnosed  every  day  in  the  US)  and  in  order  to  meet  this 
challenge  the  health  care  delivery  system  will  need  to  be  re-engineered.  This  can  become  a  reality  with  the  use 
of  the  CDMP  developed  under  this  collaborative  effort. 

BODY 

Summarized  below  is  the  originally  proposed  Statement  of  Work  (SOW)  for  reference: 

1 .  Prospective  multi  center  cost  efficiency  study  performed  using  the  JVN  Telehealth  Eye  Care  module 

2.  Prospective  multi-center  risk  benefit  study  using  the  JVN  Telehealth  Eye  Care  module 

3.  JVN  Telehealth  CDMP  program  usability  and  impact  on  clinical  workflow  study 

4.  Prospective  multi-center  clinical  outcomes  efficacy  and  cost  efficiency  study  using  the  JVN  Telehealth 
Comprehensive  Diabetes  Management  Program 

5.  Clinical  validation  of  the  Behavior  Assessment  Tool  (BAT)  developed  for  the  JVN  Telehealth  CDMP 
application 

6.  Development  and  validation  of  Learning  Level  Assessment  and  Readiness  to  Learn  tools  for  the  JVN 
Telehealth  CDMP  application 

7.  Deployment  of  JVN  Telehealth  CDMP  application  in  Tripler  Army  Medical  Center  and  Honolulu  VA  in 
Hawaii 

8.  Deployment  of  JVN  Telehealth  CDMP  application  in  VA  VISN  1  network 

9.  Deployment  of  JVN  Telehealth  CDMP  application  into  the  Department  of  Defense  TRICARE  Online 
computer  system 

10.  Establish  a  centralized  JVN  Telehealth  clinical  coordination  center  to  facilitate  the  proposed  multicenter 
clinical  trials 

11.  Clinical  validation  of  the  JVN  Eye  Care  computer  algorithm  for  automation  of  detection  of  retinal 
lesions 

12.  Clinical  validation  study  for  the  JVN  developed  retinal  imaging  device 

13.  Automation  of  the  retinal  image  taking  process  using  the  JVN  developed  retinal  imaging  device 

14.  Migration  of  JVN  Eye  Care  module  to  Microsoft  .Net  operating  platform 

15.  Development  of  additional  modules  for  the  JVN  Telehealth  CDMP  application  to  include  an  outcomes 
and  reporting  module,  an  education  scheduling  and  tracking  tool,  a  knowledge  assessment  tool,  a 
nutrition  module,  a  patient  portal  module,  integration  of  wireless  home  monitoring  devices,  and  a 
primary  care  practitioner  module 

Status  Report  for  Research  Projects  Supported  by  the  DoD  Collaborative  (SOW  #  1  to  #5) 

The  DoD  collaborative  has  8  research  projects  planned,  to  take  place  at  4  sites.  These  each  entail  testing  some  aspect  of 
the  Comprehensive  Diabetes  Management  Program  or  its  components,  namely  the  Joslin  Vision  Network  (JVN)  or  the 
Behavioral  Assessment  Tool  (BAT).  In  addition,  we  are  working  with  the  American  Institutes  for  Research  (AIR)  to 
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conduct  a  human  factors  study  of  the  CDMP;  there  is  a  research  component  to  this  work  involving  “usability  testing”  and 
an  Expert  Review  component.  The  projects  are  indicated  by  site  in  the  following  table: 


Project 

Joslin 

Hawaii 

WRAMC 

VA  Boston 

An  Assessment  of  the  Test-Retest  Reliability  of  the 
CDMP  BAT 

d 

V 

V 

An  Assessment  of  the  Validity  of  the  CDMP  BAT 

d 

V 

V 

Digital  Photography  and  Group  Discussion  as  a  Means 
of  Affecting  Dietary  Change  among  People  with  Type 

2  Diabetes:  Feasibility  Project 

V 

CDMP:  Usability  and  Impact  of  the  Workflow  on 
Diabetes  Care  Specialists  and  on  their  Process  and 
Quality  Measures 

V 

Prospective  Multi-center  Clinical  Outcomes  Efficacy 
and  Cost  Efficiency  Study  Using  the  JVN  CDMP 

V 

V 

V 

Prospective  Multi-center  Economic  Analysis  of  the 

JVN  Telehealth  Eye  Care  Module 

1 

Internet-based  Diabetes  Education  and  Case 

Management 

■■ 

Prospective  Risk  Benefit  Analysis  of  the  JVN 

Telehealth  Eye  Care  Module 

1 

Additional  Human  Factors  Study  for  the  CDMP 
Application:  Expert  Review  of  the  CDMP  (second  half) 

NA 

NA 

NA 

Additional  Human  Factors  Study  for  the  CDMP 
Application:  Usability  Testing  of  the  CDMP  (first  half) 

V 

Notes:  NA  means  not  applicable.  The  Expert  Review  of  the  CDMP  does  not  require  recruiting  subjects  and  is  done  by 
employees  of  AIR  at  their  offices. 


The  remainder  of  this  section  summarizes  the  aims  and  provides  a  status  report  of  each  project. 

An  Assessment  of  the  Test-Retest  Reliability  of  the  CDMP  BAT.  The  BAT  is  a  new,  stand-alone  screening  questionnaire 
within  the  CDMP  that  contains  questions  about  psychosocial  factors,  nutrition,  physical  activity,  alcohol  and  tobacco  use, 
medications,  general  health,  self-monitoring  of  blood  glucose  and  economic  factors.  Since  the  BAT  is  a  new  instrument,  it 
is  necessary  to  test  its  consistency  across  repeated  administrations,  i.e.,  test-retest  reliability.  The  objectives  of  this  study 
are,  first,  to  determine  the  test-retest  reliability  of  the  BAT  and,  second,  to  examine  whether  the  BAT’s  test-retest  reliability 
is  invariant  across  social-demographic  groups,  health  groups,  and  sites  participating  in  the  study.  This  is  a  multi-site 
observational  study  with  two  measurements  per  study  subject  taking  place  2  to  4  weeks  apart.  The  study  protocol  will 
involve  recruiting  42  English-speaking  individuals  from  each  site  who  are  20+  years  of  age  and  have  type  1  or  2  diabetes. 
Eligible  and  interested  participants  will  asked  to  complete  a  test  to  assess  their  executive/cognitive  function,  questions 
about  their  social-demographics,  and  the  BAT.  For  the  data  analyses,  we  will  calculate  the  correlation  between  the  two 
administrations  of  the  BAT.  Additionally,  we  will  calculate  mean  item  scores  and  mean  summary  scores  for  the  sub¬ 
sections  of  the  BAT  and  then  use  one-way  analysis  of  variance  to  examine  differences  in  the  means  among  social- 
demographic  groups,  among  health  groups  (e.g.,  differentiated  by  Ale,  BMI,  cognitive  functioning,  etc.),  and  between  the 
two  administrations  of  the  BAT  for  each  social-demographic  group  and  health  group.  Lastly,  we  will  combine  data  from 
all  of  the  study  sites  and  replicate  the  correlational  analyses. 

Status:  This  study  has  been  approved  by  the  Joslin  IRB  and  is  awaiting  approval  from  the  HSRRB.  It  is  under  review  at 
the  VA  Boston  and  will  be  submitted  to  the  IRB  at  WRAMC  shortly. 

An  Assessment  of  the  Validity  of  the  CDMP  BAT.  Similar  to  the  reliability  study  (see  above),  this  project  is  concerned  with 
the  measurement  properties  of  the  BAT;  however,  this  study  focuses  on  its  criterion,  concurrent,  and  predictive  validity. 
Concurrent  validity  is  the  correlation  between  a  measure  and  an  external  criterion  at  the  same  point  in  time.  Predictive 
validity  is  the  correlation  between  a  measure  and  an  external  future  criterion.  This  is  a  multi-site  observational  study  with 
three  measurements  per  study  participant  taking  place  6  months  apart.  We  expect  to  recruit  75  participants  from  each  site. 
Study  participants  will  be  asked  to  complete  the  BAT,  provide  anthropometries,  complete  questionnaires  and  logs/diaries 
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containing  background  and  criterion  questions.  We  will  validate  the  BAT  against  the  following  validated  instruments 
and/or  methods:  seven-day  food  diaries,  seven-day  physical  activity  diaries,  certain  physical  measures,  medical  records, 
the  Short  Form  Health  Survey  Questionnaire  (SF-36),  an  abbreviated,  validated  version  of  the  Center  for  Epidemiologic 
Studies  -  Depression  (CES-D)  scale,  the  Diabetes  Social  Support  Questionnaire  (DSSQ),  the  Summary  of  Diabetes  Self- 
care  Activities  Questionnaire  -  Revised  (SDSCA-Revised),  the  Fagerstrom  Tolerance  Questionnaire  (FTQ),  the  CAGE 
instrument,  and  several  questions  used  in  clinical  assessment  and/or  other  surveys  but  that  are  not  part  of  scales.  For  the 
analysis  of  criterion  and  concurrent  validity,  we  will  correlate  scores  from  sub-sections  of  the  BAT  with  the  scores  from 
corresponding  questionnaires,  logs,  anthropometries,  and  medical  record  data  obtained  at  approximately  the  same  time. 
Predictive  validity  will  be  evaluated  by  correlational  analysis  and  regressing  (in  multiple  regression  models)  health  factors 
(i.e.,  Ale,  BMI,  adherence  to  recommended  foot  and  eye  exams,  number  of  days  of  sick  leave,  number  of  hospitalizations, 
number  of  hospital  days,  etc.)  on  scores  from  the  sub-sections  of  the  BAT  and  (where  appropriate)  responses  to  individual 
BAT  questions  completed  at  previous  observations. 

Status:  This  study  has  been  approved  by  the  Joslin  IRB  and  is  awaiting  approval  from  the  HSRRB.  It  will  be  submitted  to 
the  IRB  at  WRAMC  shortly. 

Digital  Photography  and  Group  Discussion  as  a  Means  of  Affecting  Dietary  Change  among  People  with  Type  2  Diabetes: 
Feasibility  Project.  The  purpose  of  this  study  is  to  test  the  feasibility  of  a  new  approach  to  diabetes  nutrition  care 
involving  digital  food  photography  and  facilitated  group  discussions  of  the  digital  photographs.  The  overall  hypothesis 
regarding  this  new  approach  is  that,  by  photographing  all  meals  and  snacks  (i.e.,  keeping  ‘photo  journals’)  and 
participating  in  discussions  about  the  photographs  with  peers  and  a  nutritionist,  people  with  diabetes  will  become  aware  of 
their  behaviors  and  develop  concrete  strategies  to  meet  nutritional  recommendations.  The  study  design  is  that  of  a 
randomized  controlled  trial  with  two  groups:  one  group  will  receive  standard  diabetes  nutrition  care;  the  other  group  will 
receive  the  new  approach  to  diabetes  nutrition  care  involving  digital  photography  of  meals  and  group  discussion  of  the 
photographs  (treatment  group).  The  required  sample  size  is  36  (18  in  each  group).  All  subjects  complete  study 
questionnaires  and  a  blood  draw  before  the  digital  photography  and  discussion  sessions  begin  and  at  least  1  month  (or 
more)  after  the  digital  photography  discussion  sessions  end.  Baseline  data  will  be  evaluated  using  descriptive  analyses. 

The  hypotheses  will  be  evaluated  using  standard  comparison  tests,  such  as  two-sample  t  tests  and  ANOVA. 

Status:  Initially  this  study  was  supported  by  a  Mass  Lions  grant  to  Sven  Bursell  and  by  an  NIH  grant  to  the  GCRC  at  Joslin. 
With  this  funding,  we  drafted  a  protocol,  obtained  approval  from  the  Joslin  IRB,  enrolled  30  study  participants,  and 
completed  a  discussion  session  for  5  study  participants  in  the  treatment  group.  As  of  late  December,  however,  we  obtained 
additional  funding  support  from  the  DoD  Cooperative  Agreement.  This  was  necessary  in  order  to  pay  for  additional  digital 
cameras,  and  a  part-time  research  assistant,  among  other  things.  Since  this  change  in  sponsorship,  we  learned  that  the  DoD 
requires  that  all  studies  it  funds  obtain  approval  from  its  HSRRB.  Thus,  we  have  suspended  the  study  until  the  HSRRB 
completes  its  review. 

CDMP:  Usability  and  Impact  of  the  Workflow  on  Diabetes  Care  Specialists  and  on  their  Process  and  Quality  Measures. 
This  project  will  examine  the  usability  and  impact  on  clinical  workflow  of  the  CDMP.  Specifically,  we  will:  a)  examine 
the  Diabetes  HealtheCard  data  (which  documents  the  process  and  quality  measures  of  the  Diabetes  Quality  Improvement 
Program  (DQIP))  of  selected  diabetes  health  care  providers  in  the  Walter  Reed  Army  Health  Care  System  (WRHCS);  b) 
administer  questionnaires  regarding  aspects  of  the  diabetes  care  system  before  and  after  adoption  of  the  CDMP;  c) 
evaluate  the  use  of  the  CDMP  by  observing  the  interactions  of  these  care  providers  with  standardized  patients  (i.e.,  actors 
who  have  been  trained  to  provide  a  realistic  initial  or  follow-up  history  for  a  simulated  patent);  and  d)  use  structured  focus 
group  discussions  with  the  providers  lead  by  a  trained,  experienced  facilitator.  Health  care  providers  selected  for  this 
study  will  be  the  Nurse  Practitioners  (NPs)  of  the  Diabetes  Institute  of  the  WRHCS,  all  of  whom  have  participated  in 
developing  the  CDMP.  The  data  will  be  analyzed  qualitatively  and  quantitatively  and  summarized  by  descriptive  statistics 
and  graphical  displays.  We  will  analyze  quality  of  performance,  ease,  speed,  efficiency,  cost  effectiveness,  accuracy  and 
safety  as  a  function  of  several  design  parameters.  Where  appropriate,  we  will  use  exploratory  data  analysis,  paired  and 
unpaired  Student’s  t  tests,  chi-square  and  z  tests  for  differences  of  proportions,  analysis  of  variance,  correlation,  and 
regression  methods.  Qualitative  data  will  be  analyzed  using  nonparametric  statistics  and  using  transformations,  ranks,  or 
percentiles. 

Status:  This  study  has  received  approval  from  the  IRB  at  WRAMC.  The  HSRRB  has  deferred  review  of  it  and  granted  us 
permission  to  proceed.  Study  participants  have  been  enrolled  and  consented.  Training  in  the  use  of  the  CDMP  will  take 
place  in  May/June. 
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Prospective  Multi-center  Clinical  Outcomes  Efficacy  and  Cost  Efficiency  Study  Using  the  JVN  CDMP.  This  study  will 
determine  the  effectiveness,  safety,  and  acceptance  of  the  CDMP  combined  with  a  web-based  blood  glucose  data 
monitoring  and  computer-assisted  decision  system  (CADS).  Specifically,  it  will  determine  whether  use  of  the  CDMP  with 
CADs  is  associated  with  a)  improvements  in  glycemic  control,  b)  a  decrease  in  the  number  hypoglycemic  episodes  and 
clinic  visits,  emergency  room  visits,  hospitalizations,  and  hospital  days  for  diabetes-related  causes,  c)  a  reduction  in  the 
development  of  microvascular  complications  of  diabetic  retinopathy  and  nephropathy  or  mitigation  of  the  progression  of 
these  complications,  d)  improvement  in  lipid  profiles,  and  f)  improved  quality  of  life  for  patients.  We  have  planned  an 
open,  prospective,  randomized  clinical  trial  with  2  groups:  1  group  will  receive  standard  care,  or  no  access  to  CDMP  and 
CADS  through  their  provider;  the  other  group  will  receive  CDMP  and  CADs  through  their  provider.  The  trial  will  last  for 
two  years.  The  analysis  will  follow  the  intention-to-treat  principle  and  patients  who  received  the  CDMP  and  CADS  will 
be  compared  against  participants  w'ho  received  standard  care.  We  will  examine  group  differences  in  baseline 
characteristics  and  outcomes  using  Student’s  t  tests  and  Chi-Square  tests.  We  will  use  repeated  measures  analyses  to 
compare  changes  (e.g.,  treatment  means  or  treatment  regression  curves)  over  the  course  of  the  study.  The  analyses  will 
account  for  possible  clustering  effects. 

Status:  We  have  not  drafted  the  protocol  for  this  study  yet.  We  are  waiting  for  the  completion  of  the  usability  studies  and 
any  modifications  to  the  CDMP  that  may  result  from  it.  We  are  also  applying  for  additional  financial  support. 

Prospective  Multi-center  Economic  Analysis  of  the  JVN  Telehealth  Eye  Care  Module.  The  purpose  of  this  study  is  to 
compare  the  costs  and  cost-effectiveness  of  the  JVN  with  conventional  clinic-based  eye  examinations  among  a  diabetic ' 
cohort  receiving  annual  eye  examinations.  The  research  design  is  a  randomized  clinical  trial  that  will  provide  prospective 
data  for  insertion  into  decision  models.  In  turn,  the  decision  models  will  generate  the  data  to  evaluate  the  cost- 
effectiveness  of  the  JVN  versus  conventional  clinic-based  eye  examinations.  Consenting  patients  at  sites  of  the  Walter 
Reed  Army  Health  Care  System  (WRHCS)  with  type  1  or  type  2  diabetes  mellitus  and  scheduled  for  eye  examinations  on 
an  annual  basis  will  be  enrolled  in  the  study  and  randomized  to  conventional  clinic-based  eye  examinations  or  eye 
examinations  performed  by  the  JVN  (plus  an  assessment  of  visual  acuity).  Subjects  will  be  followed  for  one  year.  The 
study  will  track  all  costs  that  accrue  over  that  year  in  the  provision  of  care  for  both  modalities,  including  labor, 
equipment,  travel  for  the  study  subjects,  and  lost  wages/productivity  for  study  subjects,  among  others.  Cost-effectiveness 
will  be  measured  based  on  study  subjects  ’  compliance  with  the  clinical  eye  examination  and  follow-up  recommendations 
and  diagnostic  and  treatment  outcomes.  We  will  a  priori  generate  cost-effectiveness  data  based  on  diagnoses  of  diabetic 
retinopathy  and  macular  edema.  In  a  cost  consequence  analysis,  we  will  consider  other  diagnostic  outcomes  and 
outcomes  in  aggregate.  Additionally,  we  will  impute  cases  of  expected  vision  loss  and,  therefore,  project  differences  in  the 
number  of  cases  of  vision  loss  averted  between  modalities. 

Status:  We  have  completed  drafting  the  protocol  for  this  study  and  are  submitting  it  to  the  IRB  at  WRAMC. 

Internet-based  Diabetes  Education  and  Case  Management.  Patients  with  diabetes  and  elevated  HbAlc  are  at  the  greatest 
risk  for  diabetes-related  complications.  Care-management  may  be  helpful  in  this  situation,  by  providing  direct  contact 
between  high-risk  patients  and  the  healthcare  system.  This  study  will  examine  the  efficacy  and  cost-effectiveness  of  two 
methods  of  diabetes  education  and  care  management;  one  is  the  traditional  model  involving  face-to-face  encounters  and 
telephone  contact;  the  other  is  an  Internet-based  model.  We  will  compare  these  interventions  to  a  usual  care  control  group 
that  receives  no  education  or  care  management  but  is  provided  with  a  notebook  computer  and  Internet  access.  This  study 
employs  a  randomized,  prospective,  parallel  group  design  involving  patients  with  diabetes  mellitus.  Primary  outcome 
measures  include  clinical  data  (e.g.  HbAlc,  blood  pressure,  quality  of  life  questionnaires)  and  secondary  outcome 
measures  include  economic  data  (e.g.  costs  of  case  management,  medication  usage,  and  number(s)  of  ER 
visits/hospitalizations  during  the  study  period).  We  will  study  150  participants  with  elevated  HbAlc  (8.5%).  Over  12- 
months  we  will  measure  HbAlc,  office  BP,  and  scores  on  the  Problem  Areas  in  Diabetes  (PAID)  questionnaire. 
Participants  receiving  usual  care  will  receive  a  notebook  computer  and  Internet  access.  Those  assigned  to  Internet-based 
care  management  will  receive  a  notebook  computer,  Internet  access  and  will  interact  with  a  care  manager  through  a 
diabetes  education  and  care  management  website.  Those  receiving  traditional  care  management  will  interact  with  a  care 
manager  following  a  structured  contact  schedule.  Both  care  management  models  will  employ  medication  algorithms  to 
improve  glucose  and  BP  control,  with  the  secondary  goal  of  also  improving  diabetes-related  stress  and  depression.  We 
will  collect  data  on  process  measures  and  health  care  utilization  in  order  to  conduct  exploratory  analyses  on  the  cost- 
effectiveness  of  these  interventions. 

Status:  This  study  has  received  approval  from  the  VA  Boston  IRB  and  an  initial  review  from  the  HSRRB.  We  are  now 
responding  to  the  HSRRB’s  comments. 
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Prospective  Risk  Benefit  Analysis  of  the  JVN  Telehealth  Eye  Care  Module.  We  hypothesize  that  the  JVN,  supplemented 
with  a  non-invasive  eye  care  assessment  (hereafter  referred  to  as  Technology  Assisted  Ophthalmic  [TAO]  examination), 
can  substitute  for  a  complete  eye  exam  with  a  dilated  fundus  evaluation  by  an  eye  care  professional  in  patients  with 
diabetes.  To  establish  its  validity,  the  JVN  must  be  compared  prospectively  to  a  complete  eye  examination  with  dilated 
fundus  evaluation,  which  is  currently  the  standard-of-care.  Such  a  comparison  would  also  allow  for  an  economic  analysis 
of  the  JVN  in  its  ability  to  accurately  diagnose  diabetic  and  non-diabetic  eye  diseases  that  require  referral  for  eye  care.  To 
test  this  hypothesis  we  propose  the  following  objectives:  1)  To  determine  —  at  baseline  and  prospectively  over  2  years  -- 
the  accuracy  and  level  of  agreement  of  a  TAO  examination  and  a  complete  eye  examination  for  (a)  need  for  referral  to  an 
eye  care  professional  for  further  evaluation,  (b)  level  of  diabetic  retinopathy,  and  (c)  other  referable  non-diabetic  eye 
disease;  and  2)  To  perform  a  cost  minimization  analysis  comparing  the  TAO  examination  to  the  complete  eye 
examination  based  on  appropriate  referral  for  follow-up  eye  care.  We  will  study  500  patients  with  diabetes  and  at  least 
one  high-risk  characteristic  for  the  presence  and/or  progression  of  diabetic  retinopathy  using  both  a  TAO  examination  and 
a  complete  eye  examination,  to  determine  at  baseline  the  accuracy  and  level  of  agreement  between  the  two  modalities. 
High-risk  characteristics  include:  current  insulin  use,  longer  duration  of  diabetes  (>10  years),  elevated  Ale  (>8%),  renal 
disease  (serum  creatinine  >1.5  mg/dl  and/or  albuminuria),  and  elevated  blood  pressure  (systolic  BP  >160  mm  Hg).  We 
estimate  that  40%  of  the  initial  patients  will  require  early  follow-up  eye  care  based  on  findings  of  their  baseline 
examination.  Those  patients  whose  initial  diagnosis  suggests  no  significant  diabetic  retinopathy  and  no  non-diabetic  eye 
diseases  (approximately  300  patients)  will  be  followed  prospectively  over  a  2  year  period  with  an  annual  TAO  and 
complete  eye  exam  to  prospectively  determine  the  accuracy  and  level  of  agreement.  After  the  2  year  follow-up,  we  will 
perform  a  cost  minimization  analysis  comparing  the  TAO  examination  to  the  complete  eye  examination  based  on  need  for 
referral  for  eye  care. 

Status:  This  study  has  received  partial  funding  from  the  VA,  in  addition  to  the  funding  from  the  DoD  collaborative.  The 
funding  will  be  available  in  the  fall.  In  the  meantime,  we  are  submitting  the  protocol  to  the  VA  IRB  and,  once  it  is 
approved,  we  will  submit  it  to  the  HSRRB. 

Additional  Human  Factors  Study  for  the  CDMP  Application.  Although  we  believe  that  the  CDMP  can  improve  the  care- 
and  self-management  of  diabetes,  it  is  necessary  and  desirable  to  ensure  that  the  human  factors  considerations  - 
specifically  the  interface  of  the  human  and  the  machine/information  system  -  has  been  fully  optimized  to  make  the  system 
easy  to  use,  easy  to  learn,  easy  to  remember,  efficient,  and  “user  friendly”.  Thus,  we  have  subcontracted  AIR  to  conduct  a 
two-phase  human  factors  study.  In  phase  I,  human  factors  specialists  with  extensive  experience  evaluating  medical 
software  and  device  usability,  reviewed  the  existing  CDMP  software  system.  They  assessed  the  software’s  user  interface 
for  consistency  of  style  and  navigation  aids,  the  number  of  keystrokes  to  perform  frequent  tasks,  the  time  required  to 
perform  these  tasks,  and  potential  areas  of  ambiguity.  Also  in  phase  1 ,  consultants  with  expertise  in  diabetes  care 
management  will  use  the  CDMP  for  several  hours  and  provide  feedback  regarding  its  ease  of  use,  appearance,  logic, 
accuracy,  and  ability  to  facilitate  their  work  flow,  among  other  things.  After  phase  1  is  complete,  we  will  modify  the 
CDMP  accordingly  and  then  begin  phase  2  of  the  human  factors  study.  In  phase  2,  AIR  will  conduct  a  usability  laboratory 
study  with  Joslin  Diabetes  Center  health  care  providers  and  patients  with  diabetes.  The  participants,  after  providing 
informed  consent,  will  use  the  CDMP  to  perform  tasks  simulating  actual  patient  encounters  in  the  ambulatory  setting. 
Participants  will  be  video-  and/or  audio-taped  while  using  the  system,  and  will  be  administered  a  questionnaire  afterwards 
to  collect  qualitative  data  regarding  subjective  impressions.  These  results  will  be  summarized,  analyzed,  and  aggregated 
into  a  report  that  will  make  specific  recommendations  regarding  opportunities  for  improving  the  system’s  usability. 

Status:  The  first  half  of  phase  1  is  complete.  The  second  half  of  phase  1  will  begin  shortly,  depending  on  consultants' 
availability.  We  are  submitting  the  protocol  for  phase  2  to  the  Joslin  IRB.  When  it  is  approved,  we  will  submit  the 
protocol  to  HSRRB  for  their  review. 

6.  Development  and  validation  of  Learning  Level  Assessment  and  Readiness  to  Learn  tools  for  the 
JVN  Telehealth  CDMP  application 
Progress: 

These  tools  have  yet  to  be  implemented  and  have  been  placed  on  a  low  priority  by  participants  in  this  proposal. 
It  was  felt  that  available  tools  were  cumbersome  to  use  at  this  point  an  would  see  limited  utilization  by  care 
manager  users.  The  results  from  these  tools  would  not  be  used  as  part  of  the  clinical  trials.  Once  the  trials  have 
been  initiated  then  we  will  address  this  functionality  as  part  of  our  value  added  to  the  CDMP  application 
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7.  Deployment  of  JVN  Telehealth  CDMP  application  in  Tripler  Army  Medical  Center  and 
Honolulu  VA  in  Hawaii.  D.  Peters  PhD 

This  is  a  deployment  activity  planned  to  ensure  that  these  sites  come  on  line  with  respect  to  participation  in  the 
planned  multi-center  clinical  studies.  Based  on  our  experience  with  deployment  of  the  CDMP  application  at  the 
Joslin  Diabetes  Center  and  at  Walter  Read  Army  Medical  center,  we  will  have  developed  a  set  of  standard 
operating  procedures  that  will  facilitate  these  subsequent  deployments  of  the  CDMP  application. 

Of  primary  importance  is  the  integration  of  the  application  into  the  existing  Health  Information  System 
for  both  the  DoD  facility  and  the  VA  facility  at  this  site.  There  will  be  a  requirement  to  perform  the  appropriate 
data  mapping  for  integration  of  related  medical  record  information  into  the  CDMP  application.  A  utility  will  be 
developed  that  automatically  updates  patient  medical  record  information  to  the  CDMP  every  24  hours.  There 
will  also  need  to  be  a  determination  of  whether  or  not  data  collection  that  is  native  to  the  CDMP  application 
also  becomes  part  of  the  institutional  medical  record.  This  will  require  interface  between  the  IT  resources  at  the 
site  and  the  IT  resources  of  the  JVN  team. 

The  technology  requirements  are  minimal.  The  site  will  need  to  purchase  an  SQL  server  with  an  Oracle 
license  as  the  CDMP  operates  off  the  Oracle  database.  The  site  will  also  need  to  acquire  a  Weblogic  license  as 
this  is  the  operating  system  platform  for  the  CDMP.  Data  communication  will  be  over  existing  VPN 
infrastructure. 

An  evaluation  of  the  site-specific  clinical  workflow  will  be  performed  so  as  to  determine  the  optimal 
positioning  of  the  CDMP  application  within  this  process. 

Once  testing  of  the  software  integration  has  been  completed  a  series  of  training  sessions  will  be 
scheduled  for  identified  clinical  users  as  well  as  on  site  technology  people  who  will  provide  the  first  level 
support  on  technical  issues. 

Progress: 

The  original  plan  was  to  have  the  CDMP  reside  as  an  application  on  the  1CDB.  The  implementation  of  CHCS 
II,  however,  has  been  accelerated  at  TAMC  and  the  ICDB  is  no  longer  supported.  Installation  of  additional 
applications  at  TAMC  has  been  indefinitely  postponed  until  CHCS  II  has  been  fully  implemented.  As  a  result 
of  these  developments,  the  principal  investigators  believe  that  the  delay  in  installation  of  CDMP  and  the 
conduct  of  the  proposed  research  investigations  at  TAMC  will  be  unacceptably  delayed. 

This  has  resulted  in  the  modification  of  our  proposal  to  cover  the  second  and  subsequent  years  of  contract 
DAMD 17-03-2-0062  that  is  a  competing  continuation  from  the  original  contract  DAMD1 7-98-2-801 7.  We  are 
requesting  that  the  implementation  of  CDMP  and  associated  research  studies  delineated  in  the  statement  of 
work  that  were  planned  at  Tripler  Army  Medical  Center  (TAMC)  in  Honolulu,  HI  be  done  at  alternative  sites. 
The  original  plan  was  to  have  the  CDMP  reside  as  an  application  on  the  ICDB.  The  implementation  of  CHCS 
II,  however,  has  been  accelerated  at  TAMC  and  the  ICDB  is  no  longer  supported.  Installation  of  additional 
applications  at  TAMC  has  been  indefinitely  postponed  until  CHCS  II  has  been  fully  implemented.  As  a  result 
of  these  developments,  the  principal  investigators  believe  that  the  delay  in  installation  of  CDMP  and  the 
conduct  of  the  proposed  research  investigations  at  TAMC  will  be  unacceptably  delayed.  The  alternative  sites 
proposed  are:  Waianae  Coast  Community  Health  Center,  Waianae,  HI;  The  Physicians  Center  at  Mililani, 
Mililani,  HI;  and  the  Molokai  General  Hospital,  Kaunakakai,  HI.  Preliminary  site  visits  were  made  by  Dr. 
Deborah  Birkmire-Peters,  University  of  Hawaii,  Dr.  Stephanie  Fonda,  Joslin  Diabetes  Center,  and  Drew  Lewis, 
Estenda  Solutions,  Inc.  to  assess  the  suitability  of  the  sites  for  the  proposed  implementation  of  CDMP  and  the 
associated  research  studies.  They  determined  that  each  of  these  sites  was  appropriate  for  the  proposed  research 
and  the  staffs  indicated  enthusiasm  for  the  project.  The  purpose  of  this  modification  to  the  proposal  is  thus  to 
transfer  the  proposed  implementation  of  CDMP  and  associated  research  studies  delineated  in  the  statement  of 
work  that  were  planned  at  Tripler  Army  Medical  Center  (TAMC)  in  Honolulu,  HI  to  Waianae  Coast 
Community  Health  Center,  The  Physicians  Center  at  Mililani,  and  the  Molokai  General  Hospital. 

8.  Deployment  of  JVN  Telehealth  CDMP  application  in  VA  VISN  1  network.  Paul  R.  Conlin  MD 
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A  parallel  deployment  will  be  initiated  for  deployment  of  CDMP  within  the  VA  VISN  1  network.  The  same 
issues  will  be  addressed  as  outline  above  for  deployment  in  Hawaii. 

Progress; 

Deployment  of  CDMP  at  Joslin  Diabetes  Center  for  remote  access  to  the  VA  VISN  1  network  was 
completed  in  February  2005.  The  CDMP  will  be  used  to  access  data  for  the  “Internet  Based  Diabetes 
Education  and  Case  Management”  study.  The  protocol  has  been  reviewed  and  approved  by  the  local 
VA  IRB  and  is  currently  under  review  by  the  USAMRMC  Human  Subjects  Research  Review  Board 
(HSRRB). 

9.  Deployment  of  JVN  Telehealth  CDMP  application  into  the  Department  of  Defense 
HealtheForces.  Robert  Vigersky  MD,  and  Sven-Erik  Bursell  PhD 

Progress: 

Deployment  of  CDMP  into  HealtheForces  at  Walter  Reed  Army  Medical  Center  (WRAMC)  was 
completed  in  May  2005.  Diabetes  Institute  staff  at  WRAMC  received  CDMP  training  in  June  2005.  The 
Diabetes  Institute  staff  is  currently  developing  the  process  and  procedures  for  use  of  the  CDMP  in  their 
clinic.  Recent  requirements  regarding  interfacing  to  CHCSII  are  currently  being  investigated  and  a  plan 
has  been  developed  to  integrate  into  CHCS  II  as  discussed  below. 

10.  Establish  a  centralized  JVN  Telehealth  clinical  coordination  center  to  facilitate  the  proposed 
multicenter  clinical  trials.  Lloyd  M.  Aiello  and  Lloyd  P.  Aiello,  Sven-Erik  Bursell.  Co-Investigator: 
Stephanie  Fonda,  Ph.D. 

The  JVN  Telehealth  clinical  coordination  center  is  an  integral  part  of  JVN  program  moving  forward 
in  the  future  and  will  provide  the  critical  capability  of  providing  a  variety  of  professional  services  that 
provide  value  added  to  the  JVN  program  as  it  begins  to  undertake  the  proposed  rigorous  multicenter 
clinical  studies.  These  services  will  be  designed  through  the  clinical  coordination  center  under  the 
direction  of  Dr.  Lloyd  M.  Aiello  and  Dr.  Bursell  and  will  provide  direction  and  testing  of  various 
clinical  pathway  models  as  well  as  coordination  and  facilitation  of  different  JVN  related  multi-center 
clinical  trials.  The  expertise  and  programs  provided  through  the  coordination  center  will  present  a 
compelling  value  to  the  JVN  participating  sites.  The  immediate  tactical  services  that  will  need  to  be 
sustained  through  the  clinical  coordination  center  are  outlined  below: 

1.  Client  training  on  all  tools;  capture  of  clinical  feedback  for  software  updates;  modifications  to  JVN 
clinical  workflow  processes  as  desired  by  health  professionals  using  the  JVN  system. 

2.  Support  for  clinical  programs  development,  identifying  site-specific  program  requirements,  and  clinical 
pathways  and  workflows 

3.  Support  for  reading  center  services,  establishing  and  updating  certification  programs  and  courses, 
maintaining  performance  for  image  review  and  reporting  services,  scheduling  timely  quality  assurance 
reviews,  and  providing  ongoing  consultation  services 

4.  On-going  clinical  support  through  resource  development,  maintaining  quality  assurance,  consultative 
services,  and  provision  of  pre/post-deployment  periodic  site  visits 

5.  Provide  continuing  education,  both  on-site  at  the  JVN  Tele-Health  Clinical  Coordination  Center  and  at 
remote  sites  through  on  site  visits  or  video  teleconferencing 

6.  Coordinate  reading  centers  to  provide  ongoing  certification  and  re-certification  services,  for  image 
acquisition  specialists,  image  review  specialists,  and  remote  reading  centers 

7.  Coordinate  all  clinical  study  activities  associated  with  the  clinical  coordination  center  and  provide  a 
centralized  data  repository  for  the  different  proposed  clinical  studies. 

Progress: 

Development  of  the  Study  Manager  Module  in  CDMP  with  the  data  base  established  to  support  storage 
of  study  data  and  the  utilities  required  to  access  this  data  for  analysis  has  been  completed  and  code 
development  for  anonymization  of  data,  secure  messaging,  HIP  A  A  compliance,  and  centralized  storage 
has  been  completed. 
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11.  Clinical  validation  of  the  JVN  Eye  Care  computer  algorithm  for  automation  of  detection  of  retinal 
lesions.  Determine  the  sensitivity  and  specificity  and  Receiver  Operating  Characteristics  (ROC)  of  the 
performance  of  the  algorithm  for  detecting  retinal  abnormalities  from  undilated  eye  JVN  digital  video 
retinal  images  compared  to  dilated  eye  ophthamological  evaluation  of  retinal  images  from  35  mm 
retinal  fundus  photography. 

Progress:  The  algorithm  development  has  been  completed  and  undergone  validation  testing.  The  results 
of  this  study  will  be  presented  at  the  Association  for  Research  in  Vision  and  Ophthalmology  conference 
April  24  to  29  2004.  Additionally  a  manuscript  has  been  prepared  for  submission  to  SPIEE.  This 
software  module  will  be  incorporated  in  the  next  JVN  version  release  based  on  the  MS  .Net  operating 
system.  Algorithm  has  been  reworked  with  additional  improvements  to  assessment  of  overall  image 
quality  and  improved  algorithmic  detection  of  retinal  features.  Currently  the  new  algorithm  has 
undergone  training  and  validation  ans  is  currently  being  tested  against  a  new  set  of  216  JVN  retinal 
patient  studies.  Results  from  this  work  have  been  presented  at  the  ATA  meeting  in  2005  and  at  the 
ARVO  meeting  2005.  This  algorithm  will  be  implemented  into  the  JVN  version  3  release  at  the  end  of 
December  2005. 

12.  Clinical  validation  study  for  the  JVN  developed  retinal  imaging  device.  Determine  the  level  of 
agreement  in  retinopathy  diagnosis  comparing  retinal  images  taken  using  the  new  JVN  portable  retinal 
imager  and  the  clinical  gold  standard  of  dilated  eye  ETDRS  protocol  35  mm  7  stereo  standard  field 
photography. 

Progress:  This  study  has  not  been  initiated  as  retinal  prototype  development  was  slowed  due  to  reduction 
in  funding  for  2003  and  the  large  gap  between  completion  of  prior  award  and  release  of  funding  for  this 
competing  continuation.  Some  additional  work  has  occurred  but  is  a  lower  priority.  Work  will  occur 
depending  on  availability  of  funds.  Currently  modifications  to  the  prototype  now  allow  full  field  retinal 
images  to  be  acquired  without  portions  of  retina  being  occluded  by  reflections  from  the  lens  and  cornea. 
Currently  we  are  pursuing  negotiations  with  an  ophthalmic  company  to  productize  the  retinal  imaging 
system.  Examples  of  retinal  images  obtained  by  this  prototype  are  shown  below. 


Figure  1:  Retinal  image  obtained  from  a  young  normal  non-diabetic  male  volunteer 
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Figure  2.  Retinal  image  obtained  from  a  60  year  old  male  non-diabetic  volunteer 

13.  Automation  of  the  retinal  image  taking  process  using  the  JVN  developed  retinal  imaging  device. 

Development  of  scanning  technology  that  will  automate  imaging  and  simultaneous  stereo  acquisition  of 
different  regions  of  the  retina  determined  to  be  essential  for  accurate  diagnosis  of  diabetic  retinopathy. 

Progress:  This  component  has  not  been  initiated  due  to  funding  reduction  in  2003  and  the  large  gap 
between  completion  of  prior  award  and  release  of  funding  for  this  competing  continuation.  Once  the 
retinal  imaging  device  can  be  produced  we  will  conduct  the  necessary  clinical  validation  study. 


Software  Application  for  3-Dimensional  Visualization  of  the  Retina 
Optic  Nerve  Head  Mapping  from  Stereo  Pairs 

In  lieu  of  the  above  automation  process  we  have  been  investigating  utilization  of  image  analysis 
techniques  to  provide  three  dimensional  mappings  of  the  retina.  These  would  be  of  value  to  provide  an 
indication  of  relative  elevation  in  the  macula  as  an  indication  of  risk  for  macula  edema  a  sight 
threatening  process  or  development  of  optic  disc  changes  potentially  associated  with  the  development  of 
glaucoma. 

The  figures  below  illustrate  the  work  completed  in  this  area. 
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Figure  3.  Macula  3-Dimensional  reconstruction 


14.  Migration  of  JVN  Eye  Care  module  to  Microsoft  .Net  operating  platform.  Development  of  a 
software  enhancement  that  will  allow  improvement  in  performance  and  reduce  costs  of  technological 
support. 

Progress:  The  software  development  for  the  JVN  image  acquisition  application  has  been  completed  and  is 
currently  undergoing  user  acceptance  testing  at  the  Joslin  Diabetes  Center.  Additional  programming  needs  to  be 
completed  for  the  JVN  Diagnostic  Display  work  station  that  is  scheduled  for  completion  in  July  2004.  In  the 
interim  all  images  captured  on  the  JVN  v3.0  software  can  be  viewed  on  JVN  v  2.2  Diagnostic  Display 
workstations.  The  JVN  image  acquisition  software  has  been  further  modified  to  allow  integration  of  any  retinal 
imaging  system  into  the  application.  This  has  required  reconfiguration  of  the  acquired  images  so  that  they  can 
still  be  viewed  on  JVN  diagnostic  display  version  2  work  stations. 

CHCSII  Integration  Requirement. 

Because  of  the  introduction  of  CHCS  II  in  the  WRAMC  Network  we  have  also  initiated  an  effort  to  accelerate 
the  development  of  the  JVN  diagnostic  display  work  station  such  that  the  CDMP  and  JVN  eye  care  are  a  single 
application  that  can  be  embedded  directly  into  CHCS  II.  This  effort  involves  the  integration  of  JVN  diagnostic 
display  into  the  CDMP  application.  It  requires  the  development  of  a  DICOM  viewer  integrated  as  a  module  in 
CDMP  and  leverages  all  the  reporting  capabilities  of  CDMP  to  generate  diagnostic  findings,  clinical,  diagnosis 
and  clinical  performance  reporting.  Included  in  this  design  will  also  be  the  ability  to  provide  assessments  and 
diagnosis  for  general  ophthalmology  applications.  The  advantage  here  is  that  the  whole  application  exists  on  an 
open  architecture  leveraging  the  web  based  technology  used  for  CDMP  and  no  longer  requires  reliance  on  Agfa 
as  a  third  party  vendor  for  the  PACS  system.  This  is  a  necessity  as  Agfa  has  been  no  responsive  to  our  support 
requests  over  the  past  1 8  months  despite  the  fact  that  we  have  paid  them  $30,000  per  year  for  support  of  our 
application.  The  Figure  below  illustrates  the  high  level  architecture  for  the  proposed  solution. 


We  have  completed  the  functional  requirements  definition  portion  of  the  project  and  currently  our  in  house  JVN 
team  and  Estenda  are  working  through  the  technical  design  portion  of  the  effort.  We  expect  to  have  available 
for  beta  testing  amongst  the  participating  study  groups  by  the  end  of  December  2005. 
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In  parallel  with  our  in  house  development  effort  here  we  will  also  work  with  CHCS  II  developers  to  initiate  the 
integration  of  the  JVN  application  into  CHCS  II  in  such  a  way  that  the  proposed  studies  at  WRAMC  are  not 
compromised.  The  planned  work  will  be  accomplished  in  3  phases  as  outlined  below: 

Phase  1 :  CDMP  entry  exists  as  an  icon  on  the  CHCSII  web  portal.  CDMP  is  still  connected  to  ICDB  (Integrated 
Clinical  Data  Base)  at  WRAMC.  This  will  allow  us  to  continue  our  studies  in  the  short  term. 

Phase  2:  We  will  establish  connection  to  the  CHCSII  data  repository  (CDR)  that  will  take  over  ICDB  function 
for  CHCSII.  This  provides  a  more  direct  connect  between  CDMP  and  CHCSII. 

Phase  3:  Integration  of  CDMP  into  CHCSII,  establishment  of  the  CHCSII  data  warehouse  by  CHCS  II 
development  (CDM),  removal  of  CDMP  database  server  with  all  data  now  going  to  CHCSII  CDM.  At  this  point 
we  have  total  integration  with  CHCS  II. 

15.  Development  of  additional  modules  for  the  JVN  Telehealth  CDMP  application.  These  modules  are 
technology  enhancements  to  the  existing  CDMP  application  and  will  include  an  outcomes  and  reporting 
module,  an  education  scheduling  and  tracking  tool,  a  knowledge  assessment  tool,  a  nutrition  module,  a 
patient  portal  module,  integration  of  wireless  home  monitoring  devices,  and  a  primary  care  practitioner 
module. 

Progress:  Comprehensive  Diabetes  Management  Program  (CDMP) 
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For  the  last  year  and  a  half  our  we  have  divided  our  time  -  developing  new  or  deepening  existing  modules, 
deploying  CDMP  in  a  variety  of  underserved  communities,  and  creating  and  delivering  user  help  files  and 
training  materials. 

Overall,  we  have  been  able  to  capitalize  on  our  original  mission:  Create  a  patient-centric  diabetes  treatment 
application  that  fills  the  gaps  in  standard  disease  management  software  and  empowers  patients  to  become  good 
self-managers  through  timely,  accurate  information  and  solid,  collaborative  relationships  with  their  care 
providers.  Support  those  providers  with  a  number  of  clinical  decision  support  tools  and  embedded  education. 

We  are  fortunate  to  be  working  with  our  original  development  team,  using  a  strict  system  development 
methodology,  conscious  of  the  need  for  quality  and  documentation. 

New  or  Updated  Features  -  in  reverse  chronological  order: 

o  Reporting  Module  -  CDMP  is  establishing  a  set  of  standard  reports  to  be  embedded  and,  therefore, 
available  to  every  user.  The  first  group  of  reports  will  fulfill  DQIP,  HEDIS  and  other  certification 
criteria.  Later  reports  will  encompass  more  patient,  provider,  and  site  performance. 

o  Study  Manager  -  Developed  to  aid  performance  reporting  and  outcomes,  this  application  has  generated 
much  interest  in  the  research  and  provider  communities  because  it  lays  out  the  process  of  capturing  that 
information  in  a  series  of  successive  steps. 

o  Nutrition  Module  -  Bi-level  nutrition  assessment  tools,  the  first  for  diabetes  generalists  to  assess 

patients  for  the  amount  of  control  they  have  over  the  food  issues  in  their  lives.  The  second  assessment  is 
for  CDEs  and  nutritionists  to  assess  more  subtle  financial,  emotional  and  lifestyle  issues  while  looking  at 
food  choices  and  control. 

We  have  begun  the  process  of  folding  into  the  module  an  application  that  analyzes  the  selected 
components  of  foods  and,  with  a  nutrition  profile  of  a  patient,  evaluates  the  balance  of  the  patient’s 
meals  and  makes  suggestions  that  include  essential  nutrients  and  foods. 

o  Patient  Portal  -  DME  Everywhere,  our  portal,  is  evolving  with  rigorous  use  by  the  VA.  We  have  made 
significant  changes  in  language  level  and  visual  clarity  in  the  past  six  months.  The  portal  allows  the 
patient  access  to  a  current  CarePlan,  health  profile  with  underlying  clinical  guidelines,  and  vetted  web 
and  other  media-based  materials  for  a  Learning  Plan.  The  following  is  a  partial  list  of  benefits  for  the 
patient: 

o  Uploading  and  trending  reports  for  self-monitoring  information  -  BG,  BP  and  other  monitors 
o  Self-management  advice 
o  Wellbeing  and  coping  surveys 

o  Lab  test  results  annotated  with  patient  language  guidelines 
o  Graphical  indicators  of  Vitals  -  Ale,  weight,  BP 

o  Graphical  Health  Profile  -  With  risk  level  (red,  yellow,  green),  assessment  taken  from  Risk 
Profile  along  with  “How  to  Lower  Risk”  short  narrative  in  patient-friendly  terms 
o  Patient  Take-Home  version  of  CarePlan  -  same  as  the  patient  is  given  at  a  visit 
o  Mechanisms  to  communicate  securely  with  CareTeam  on  routine  issues  or  problems,  such  as 
adjustments  to  self-management  plan,  resource  information  from  their  care  team,  non-urgent 
questions 

o  CarePlan  -  Significantly  updated  dynamic  care  planning  is  done  and  adjusted  with  the  patient.  The  plan 
addresses  our  three  targets  -  physical  wellness,  lifestyle  self-  management,  and  psycho-social  health  - 
while  considering  the  issues  of  provider  time,  data-entry  redundancy  and  plain  language.  CarePlans  are 
easily  updated  and  created. 

o  Survey  Architecture  -  Our  robust  architecture  has  allowed  us  to  add  surveys  of  all  types  to  the  core 
application  to  assess  for  depression,  willingness  to  work  on  problem  areas,  and  education  needs. 
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Training 

With  a  growing  user  base  and  more  and  more  requests  for  demos,  we  have  developed  both 

•  demo  systems  with  scripts  that  take  the  user  through  the  core  application,  DME  and  the  newer 
assessments  and 

•  training  materials  that  familiarize  new  users  first,  with  the  basic  features  -  Alerts  and  Reminders,  the 
Patient  Snapshot,  the  Risk  Profile,  and  CarePlanning  -  then  gradually  add  in  nuance  and  greater 
detail  that  optimize  the  application. 

CDMP  Clinical  Data  Warehouse  -  CDMP  was  designed  to  collect  and  maintain  large  amounts  of  data  and 
a  structure  able  to 

•  de-identify  that  data, 

•  perform  structured  queries, 

•  provide  studies  support,  and 

•  develop  predictive  models. 

Expert  Reviews 

The  CDMP  and  its  behavioral  survey  tool,  Understanding  Your  Diabetes,  have  undergone  expert  reviews 
from  American  Institutes  for  Research.  It  is  currently  undergoing  usability  tests  with  the  same  organization. 

Our  own  semi-annual  reviews  with  the  development  consortium  have  produced  pragmatic  and  valuable 
insights  leading  to  more  depth  in  the  tools  and  greater  interest  from  potential  users. 


KEY  RESEARCH  ACCOMPISHMENTS 

•  Development  of  study  design  and  protocols  for  the  proposed  multicenter  clinical  studies 

•  Refinement  of  the  automated  retinal  image  analysis  algorithm  to  improve  sensitivity  and  specificity  for  detecting 
microaneurisms 

•  Retinal  imaging  prototype  is  now  acquiring  good  quality  retinal  images 

•  Study  management  module  for  CDMP  completed  and  ready  to  be  used  for  start  of  clinical  trials 

•  CDMP  patient  portal  completed  and  deployed  with  CDMP 

•  Validation  of  retinal  imaging  application  for  detection  of  non-diabetic  retinal  findings 

•  Algorithm  development  for  detecting  elevated  retinal  features  has  been  completed  and  will  undergo  clinical 
testing  with  comparisons  to  OCT  measurements  of  retinal  thickness 
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Association  Telehealth  Practice  Recommendations  for  Diabetic  Retinopathy 
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CONCLUSIONS 

The  study  design  for  the  proposed  multicenter  prospective  clinical  trials  have  been  completed,  have  received 
approval  though  participating  organization  human  subjects  review  boards  and  are  awaiting  final  formal 
approval  from  HSRRB  at  USMRMC.  These  prospective  studies  are  important  as  they  are  designed  to 
demonstrate  both  clinical  efficacy  and  cost  effectiveness.  If  the  hypothesis  of  clinical  efficacy  and  cost 
effectiveness  are  born  out  by  the  results  from  these  trials  then  the  broad  introduction  of  this  application  into  the 
health  care  system  will  be  facilitated  and  could  result  in  significant  health  care  dollar  savings  associated  with 
the  care  of  patients  with  diabetes. 

The  Comprehensive  diabetes  management  program  (CDMP)  application  has  undergone  a  number  of  versions 
and  we  have  developed  significant  new  functionality  to  the  application.  We  have  completed  the  study 
management  module.  This  module  facilitates  the  management  of  any  clinical  research  study  using  an  electronic 
platform  with  a  centralized  study  data  repository.  This  will  facilitate  study  management  and  subsequent  data 
analysis.  The  time  savings  on  the  study  management  component  is  expected  to  be  significant  and  results  from 
the  use  of  this  system  could  be  used  to  demonstrate  a  broader  application  of  this  module  for  any  randomized 
clinical  trial  such  as  a  drug  study  that  needs  to  be  performed  by  a  pharmaceutical  company. 

The  other  significant  effort  is  the  completion  of  the  CDMP  patient  portal  (Diabetes  Management  Everywhere 
(DME).  The  DME  portal  facilitates  secure  communication  between  patient  and  provider  with  respect  to 
downloaded  patient  information  such  as  weight,  blood  pressure,  and  blood  glucose  monitoring  values.  The 
system  also  allows  patients  to  down  load  digital  images  for  review  by  care  managers,  for  example  patients  can 
down  load  images  of  their  meals  for  discussion  regarding  portion  size  and  calories  with  nutritionists. 

The  JVN  eye  care  component  is  being  integrated  into  the  CDMP  application.  This  is  significant  as  we  can  now 
leverage  the  reporting  power  of  CDMP  to  provide  performance  reporting  and  quality  assurance.  Additionally, 
the  use  of  the  CDMP  now  means  that  we  no  longer  need  to  deploy  multiple  servers  for  the  JVN  application 
resulting  in  significant  hardware  savings  as  well  as  support  savings.  Finally,  integration  of  JVN  eye  care  with 
the  CDMP  application  allows  ultimate  embedding  of  the  application  into  CHCS  II  and  VA  Vista  systems. 

Results  fro  the  JVN  eye  care  studies  have  shown  that  the  JVN  usage  for  retinal  imaging  through  non  dilated 
pupils  is  equivalent  to  both  dilated  eye  photography  and  a  dilated  eye  exam  performed  by  a  retinal  specialist 
(both  current  clinical  gold  standards).  We  have  also  enhanced  the  retinal  imaging  application  to  include  clinical 


guidelines  based  management  of  eye  disease  by  including  systemic  risk  factors  available  through  the  CDMP 
application. 


20 


Appendix  1:  Presented  ARVO  2005 


Appendix  2:  Presented  ATA  2005 
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Abstract 

This  paper  presents  the  effects  of  image  quality,  given  by  the  number  of  pixels  used  to  define  the  image.  A 
microaneurysm  (MA)  segmentation  algorithm  that  has  been  shown  to  achieve  about  90%  sensitivity  and  specificity  for 
clinical  classification  using  high  resolution  images  of  those  diabetic  patients  who  present  with  MAs  was  applied  to  low 
resolution  images  to  assess  the  effects  of  lower  resolution  of  sensitivity  and  specificity.  The  low  resolution  (640  by  480 
pixels)  45°  field  of  view  (FOV)  was  provided  by  a  non-mydriatic  camera.  High  resolutions  images  from  a  mydriatic 
fundus  camera  were  acquired  by  digitizing  35mm  color  film  slides  to  1400  x  1200  pixels  for  a  30°  FOV.  The  image 
quality  of  the  digitized  35mm  images  was  considerably  better  than  those  from  the  non-mydriatic  camera.  Segmentation  of 
microaneurysms  (MAs)  was  performed  using  the  green  channel  image  of  the  two  modalities.  The  images  were  contrast 
enhanced  and  corrected  for  uneven  illumination.  The  images  were  then  filtered  using  a  tophat  morphological  filter  and  a 
threshold  applied  to  segment  the  candidate  MAs.  Because  the  retinal  vessels  are  similar  in  intensity  and  contrast  to  MAs, 
the  retinal  vasculature  was  segmented  using  matched  filters  to  remove  the  vessel  artifacts  from  the  image.  Ground  truth, 
which  was  provided  by  an  ophthalmic  analyst,  was  used  in  the  tuning  step  to  find  the  bounds  of  the  different  intensity  and 
shape  features  that  characterize  the  MAs.  These  features  were  used  to  distinguish  between  MAs  from  other  artifactual 
objects  on  the  image.  The  best  result  that  could  be  achieved  with  the  lower  resolution  images  was  70%  sensitivity  and 
specificity.  We  have  found  that  the  effects  of pixel  resolution  on  an  automated  segmentation  routine  to  be  of  significant  in 
obtaining  higher  sensitivity  and  specificity, 

1.  Significance 

Diabetes  has  reached  epidemic  levels  in  the  developed  countries.  Retinopathy  is  one  of  the  most  common  complications 
of  diabetes,  which  if  not  managed  appropriately  can  lead  to  vision  loss  and  blindness.  There  is  a  need  for  early  diagnosis 
and  appropriate  management  and  treatment  of  diabetic  retinopathy  and  macula  edema  by  retinopathy  assessment  in  the 
diabetic  population  at  regular  intervals  as  recommended  through  standard  practice  guidelines.  In  present  health  care 
systems,  screening  of  retinal  images  for  diabetic  retinopathy  is  limited  by  its  extensive  need  for  human  resources.  An 
automated  computer  based  system  for  base  line  retinopathy  assessmentwill  aid  in  reducing  the  future  economic  impact  on 
health  care  professionals  and  ensuring  proper  treatment  for  those  patients  where  timely  diagnosis  could  prevent  blindness. 


2.  Related  Work 

A  number  of  research  efforts  in  screening  patients  for  diabetic  retinopathy  using  automatic  segmentation  of  MAs  have 
been  previously  reported.  Larsen  et  al.  (2003)  [9,  10],  evaluated  the  performance  of  an  automated  fundus  photographic 
image  analysis  algorithm  in  high-sensitivity  and/or  high  specificity  classification  of  patients  with  diabetes  with 
undiagnosed  DR  from  those  without  retinopathy.  In  their  study,  the  data  set  consisted  of  260  diabetic  patients  of  which 
137  presented  with  DR.  The  images  were  taken  with  a  60  degree  FOV  mydriatic  camera  centered  on  the  macula  and  the 
image  size  was  1947x1296  after  digitization  of  the  35mm  slides.  When  adjusted  to  high  sensitivity,  the  automated  system 
demonstrated  sensitivity  at  0.93  and  specificity  at  0.72.  When  adjusted  to  high  specificity  the  automated  system 
demonstrated  a  sensitivity  of  0.76  and  specificity  at  0.97.  Their  approach  limited  significantly  the  regions  of  interest  that 
were  processed  for  retinopathy  screening.  Their  paper  did  not  describe  the  issue  of  image  quality,  i.e.  how  the  images  that 
were  processed  were  selected.  In  addition  to  image  quality,  other  criteria  such  as  patient  selection  and  regions  of  interest 
were  not  described. 


Spencer  et  al.  (1996)  [3],  described  an  image  processing  strategy  for  the  segmentation  and 
quantification  of  MAs  in  fluorescein  angiograms  (FAs)  of  the  ocular  fundus.  A  total  of  only  4  images  with 
many  micro-aneurysms  (>20  per  image)  were  used  for  the  validation.  A  maximum  value  of  0.82 
sensitivity  was  achieved  by  the  computer’s  segmentation  with  approximately  250  false  positive  MAs 
detected  in  all  4  images.  Since  the  true  negatives  are  undefined,  the  specificity  cannot  be  calculated. 

The  most  significant  observation  of  their  study  in  relation  to  this  study  is  their  use  of  fluorescein 
angiograph  images.  FA  images  are  significantly  higher  contrast  and  are  therefore  would  expect  to 
achieve  higher  sensitivity  in  detecting  and  segmenting  MAs. 

Yang  et  al.  (2001)  [2],  described  an  algorithm  for  detecting  MAs  in  low  resolution  (20  microns  per  pixel) 
color  fundus  images.  The  algorithm  is  specifically  designed  for  low-resolution  images  in  order  that  it 
may  be  used  in  low-cost  ophthalmic  imaging  systems  for  mass  screening  program.  Using  images  from 
three  normal  retinas,  Yang  et  al.  inserted  hundreds  of  artificial  MAs  into  the  images  to  produce  a 
ground  truth  for  measuring  the  performance  of  their  algorithms.  Unfortunately,  the  size  distribution  and 
contrast  of  these  artificial  MAs  was  not  given.  Yang  et  al.  tested  the  algorithm  on  46  images  from  a 
Canon  non-mydriatic  color  fundus  camera  (of  normal  and  diabetic  patients)  and  achieved  a  global 
clinical  sensitivity  of  0.90  (the  ability  to  detect  at  least  one  MA  in  an  eye  with  diabetic  retinopathy)  and 
0.80  global  clinical  specificity  (the  ability  of  not  detecting  any  MAs  in  a  healthy  eye).  Though  the  results 
reported  are  better  than  ours,  they  used  a  smaller  data  set  from  a  single  field  to  evaluate  their 
performance  and  detailed  results  were  not  reported.  Further,  the  ROC  curve  in  their  study  shows  a  very 
high  rate  of  false  positives  at  90%  sensitivity.  Again,  there  is  no  description  on  how  the  subjects  for  the 
image  set  were  selected.  A  bias  can  be  introduced  by  not  selecting  subjects  randomly  to  reflect  the 
spectrum  of  image  quality  as  affected  by  an  individual’s  pupil  diameter,  clarity  of  the  anterior  segment, 
and  pigmentation. 

Philips  et  al.  [1]  reported  on  quantification  of  diabetic  maculopathy  by  digital  imaging  of  the  fundus. 

They  used  FAs  to  segment  and  count  MAs.  A  system  called  ADRIS  [12]  (an  automatic  diabetic  retinal 
image  screening  system)  was  cited  in  the  literature  and  was  used  to  screen  patients  for  eye  diseases 
where  the  focus  was  on  optic  cup  to  disc  ratio,  presence  of  exudates  and  tortuous  blood  vessels. 

The  literature  review  supports  a  number  of  conclusions: 

1 .  Improved  sensitivity  and  specificity  are  correlated  to  the  resolution  of  the  digital  images  [9]. 

2.  Performance  of  computer-aided  screening  of  digital  images  improves  when  pupil  dilation  is  used 
in  the  photography  [9]. 

3.  There  is  no  published  evidence  that  a  robust  computer-aided  screening  system  for  diabetic 
retinopathy  exists. 

4.  Published  studies  have  not  explained  how  to  manage  multiple  fields  in  determining  the  clinical 
classification  of  DR  for  an  eye. 

3.  Dataset 

Two  different  data  sets  were  used  to  test  the  screening  system.  Low  resolution  images  were  collected  using  a  45°  FOV 
(640x480)  stereo  non-mydriatic  digital  video  color  camera  (Topcon  TRC  NW-5S,  Paramus,  NJ).  The  data  was  provided 
by  the  Joslin  Diabetes  Center  and  are  described  by  Bursell,  et  al.  [4].  Three  different  fields;  one  centered  on  the  macula, 
one  on  the  superior  temporal  region  and  one  on  nasal  were  captured  for  each  patient  [4],  Training  was  performed  on 
retinal  images  from  97  subjects  (74  with  No  Diabetic  Retinopathy  (NDR)  and  23  with  diabetic  Retinopathy  (DR)).  A 
validation  was  performed  on  an  independent  data  set  (non-mydriatic)  of  91  subjects  (56  NDR  and  35  DR).  Images  of  this 
dataset  were  screened  by  an  algorithm  that  used  Fourier  power  spectral  density  as  a  metric  for  image  quality.  High 
resolution  images  were  collected  from  35mm  color  images  with  a  30°  FOV  through  dilated  eyes  with  a  standard  fundus 
camera.  A  total  of  18  subjects  (10  DR  and  8  NDR)  were  used  in  this  study.  Images  of  this  dataset  were  selected  by  an 
expert  retinal  grader. 


4.  Methods 


The  computer-based  digital  retinal  photo  screening  system  first  identifies  automatically  those  digital  fundus  images  of 
sufficient  quality  for  the  automatic  segmentation  of  MAs  by  calculating  the  Fourier  power  spectral  density.  Those 
photographs  that  do  not  meet  a  minimum  image  quality  standard  are  referred  to  the  human  grader  for  manual  reading. 
The  images  that  passed  the  quality  test  were  then  corrected  for  non-uniform  illumination  using  a  Gaussian  filter  [5], 
Applying  the  Gaussian  filter  to  an  image  results  in  a  low-pass  filtered  image  that  represents  the  global  lighting 
distribution.  This  low-pass  filtered  image  is  subtracted  from  the  original  image  to  produce  an  image  that  appears  to  be 
more  evenly  illuminated.  Uneven  lighting  is  common  to  most  retinal  images,  especially  when  illuminated  non- 
mydriatically. 

The  MAs  were  segmented  using  a  top-hat  transformation  [3],  The  segmented  image  was  then  binarized.  Segmentation  of 
the  retinal  vessels  was  performed  using  a  Gaussian-matched  filter  [6,  7].  The  resulting  retinal  vessel  image  was  used  as  a 
mask  to  remove  noise  in  the  MA-segmented  image  that  was  due  to  retinal  vessel  residue  from  the  top-hat  transformation 
intended  to  segment  only  the  MAs.  The  top  hat  transformation  can  and  usually  does  segment  a  number  of  objects  that  are 
not  MAs,  i.e.  false  positives.  The  number  of  false  positives  is  minimized  by  classifying  the  objects  according  to  features 
(such  as  shape  and  color)  that  discriminate  MAs  from  the  general  assortment  of  false  positive  objects.  The  optimal  range 
of  values  for  features  that  represent  MAs  were  found  using  a  search  algorithm  where  the  goal  was  to  maximize  the 
number  of  true  positives  (objects  that  are  MAs)  and  minimize  the  false  positives  (objects  that  are  not  MAs)  as  manually 
defined  by  an  ophthalmic  technician  (grader). 

Since  there  are  low  resolution  images  for  three  fields  for  each  patient,  the  information  (segmented  objects)  from  the  three 
fields  was  combined  to  yield  the  clinical  classification  (normal  or  DR).  Each  segmented  image  contained  objects  there 
were  classified  by  the  analyst  as  true  positives,  but  there  remained  a  certain  number  of  false  positive  objects.  Our 
approach  for  producing  a  clinical  diagnosis  was  based  on  the  statistical  probability  that  the  number  of  objects  in  a 
segmented  image  that  were  true  positives  would  increase  as  the  total  number  of  segmented  objects  increased. 
Additionally,  it  was  believed  that  the  statistical  distribution  of  true  positives  and  false  negatives  varied  from  field  to  field. 
A  genetic  algorithm  [8]  was  used  to  determine  the  optimal  weighting  to  apply  to  the  objects  in  each  of  the  three  fields  and 
to  discover  an  optimal  threshold  that  yields  the  greatest  clinical  sensitivity  and  specificity. 

This  final  step  in  the  screening  process  makes  this  system  differ  from  those  previously  developed  and  cited  in  the 
literature  [9,  10].  The  screening  system  implemented  a  technique  that  made  it  unnecessary  to  detect  automatically  all 
MAs  and  to  reject  all  other  segmented  objects  that  resemble  MAs  in  an  image.  To  screen  the  segmented  digital  images,  a 
threshold  for  objects  (true  positives  and  false  positives)  was  determined  statistically  using  a  database  of  retrospectively  analyzed 
data,  such  that  a  digital  image  presenting  with  objects  greater  than  a  critical  threshold  is  classified  as  one  coming  from  a  patient  with 
diabetic  retinopathy.  Figure  1  illustrates  this  concept  where  images  having  a  number  of  segmented  objects  greater  than  the 
critical  threshold  (8  objects)  are  classified  as  a  patient  with  diabetic  retinopathy.  In  this  illustration  an  image  with  6  to  1 1  segmented 
objects  can  be  either  normal  or  diabetic  retinopathy.  There  is  a  trade-space  that  determines  the  sensitivity  and  specificity 
depending  on  the  selection  of  the  critical  threshold.  A  receiver  operating  characteristic  (ROC)  curve  is  constructed  by 
varying  the  threshold,  allowing  the  user  to  select  a  desired  screening  sensitivity  and  specificity.  In  this  approach,  the 
process  does  not  attempt  to  maximize  the  clinical  sensitivity  based  on  detecting  all  or  most  of  the  MAs  in  any  of  the  given 
fields,  but  rather  optimizes  the  clinical  sensitivity  and  specificity  by  using  statistical  information  on  the  number  of  objects 
that  are  segmented. 
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Figure  1.  Illustration  of  how  a  threshold  for  the  number  of  objects  detected  determines  the  true  and 
false  positive  and  true  and  false  negative  rates.  For  the  threshold  <  8,  those  eyes  with  less  than  or 
equal  to  8  objects  would  be  classified  as  normal;  however,  the  distribution  of  diabetic  retinopathy  eyes 
with  6  to  8  objects  is  not  zero.  Those  eyes  could  be  miss-classified  as  normal.  Conversely,  there  are  a 
few  cases  where  normal  eyes  have  9  to  10  objects.  Those  eyes  would  be  mis-classified  as  diabetic 
retinopathy,  but  could  actually  be  normal. 


5.  Results 

5.1.  High  Resolution  Fundus  Images 

The  results  presented  here  are  from  a  training  dataset  that  included  10  abnormal  and  8  normal  subjects  with  only  one  field. 
A  clinical  sensitivity  of  90%  and  a  specificity  of  87.5%  were  achieved  with  a  threshold  of  6  objects.  Figure  2  gives  the 
results  on  segmentation  from  high  resolution  images.  Though  the  dataset  is  very  small  and  no  validation  was  performed 
with  an  independent  dataset,  we  could  segment  more  MA’s  and  minimize  the  number  of  FP’s  significantly  on  the  normal 
images  and  achieve  a  high  clinical  sensitivity  and  specificity. 

5.2.  Low  Resolution  Fundus  Images 

Our  goal  was  to  determine  the  maximum  clinical  sensitivity  and  specificity  that  is  achievable  for  the  low  resolution 
fundus  images.  For  the  training  set,  we  achieved  a  clinical  sensitivity  of  83%  and  a  specificity  of  69%.  For  the 
independent  validation  set,  we  achieved  a  clinical  sensitivity  of  69%  and  a  specificity  of  66%  with  a  weighted  threshold  of 
2  objects  by  combining  all  three  fields.  The  prevalence  of  normal  and  abnormal  patients  in  training  is  0.24  and  0.76 
respectively.  The  prevalence  of  normal  and  abnormal  patients  in  validation  is  0.39  and  0.62  respectively.  The  positive 
predictive  value  and  the  negative  predictive  value  were  found  to  be  greater  than  the  prevalence  for  normal  and  abnormal 
in  both  training  and  validation.  This  shows  that  the  algorithms  is  always  better  than  chance  in  detecting  abnormal/normal 
patients 


Conclusion 

There  are  medical  and 
technical  reasons  why  one 
would  prefer  to  have  low 
resolution  images  collected 
on  non-mydriatic  cameras 
for  broad  scale  screening  of 
the  at-risk  population. 
From  a  medical  perspective, 
a  non-mydriatic  fundus 
camera  would  allow  retinal 
imaging  without  the  need 
for  dilating  eye  drops 
(Lower  light  levels  for 
imaging  ease  of  use  for 
imagers,  and  improved 
patient  compliance). 

However,  the  non-mydriatic 
fundus  camera  produces 
images  with  less  contrast 
and  poorer  over  all  lighting 
than  the  mydriatic  camera 

where  the  subjects’  eyes  have  been  dilated.  There  are  technical  motivations  for  lower  resolution  images.  For 
telemedicine  applications,  transmission  and  storage  of  large  images  can  result  in  bottleneck  in  the  system.  Implementers 
of  tele-health  systems  want  to  keep  the  bandwidth  requirements  for  transmitting  data  as  low  as  possible  to  reduce  costs. 

Our  study  has  shown  that  reduced  image  quality,  including  lower  resolution,  will  result  in  lower  sensitivity  and 
specificity  in  the  clinical  classification  of  patients  with  MAs.  The  study  showed  only  two  extreme  image  quality  levels. 
In  the  future,  a  study  is  needed  that  shows  whether  the  loss  of  sensitivity  and  specificity  is  linear  or  non-linear  in 
progressing  from  90%  to  70%.  One  would  like  to  know  what  aspects  of  image  quality  are  the  most  important,  e.g. 
contrast  or  resolution.  Image  compression  may  also  need  to  be  studied  in  terms  of  its  effect  on  sensitivity  and  specificity. 
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Figure  2.  MA  segmentation  of  high  resolution  images.  The  true  positives 
and  false  negatives  were  identified  on  the  image.  There  are  no  false 
positives  segmented  on  this  image. 


